Integrating Many Techniques for Discovering Structure in Data

This paper describes a formal representation of the discovery process that e ciently integrates of any number of data analysis strategies, regardless of their similarities and di erences. We have implemented a system based on this formalization, called the Scientist's Empirical Assistant (SEA). SEA employs several analysis strategies from the discovery literature, including techniques for function nding, causal modeling, and Bayesian conditioning. It uses high-level knowledge about the discovery process, the strategies, and the domain of study to coordinate the selection and application of analyses. It relies on the skills and initiatives of an expert user to guide its search for structure. Finally, it designs and runs experiments with a simulator to verify its ndings. SEA is currently capable of performing a full cycle of discovery and analysis, from hypothesis formulation to experiment design, data collection, analysis, and the generation of explanatory hypotheses. In addition, the formalization on which it is based provides an environment for studying the how, what, why, and when of intelligent data analysis. Integrating Many Techniques for Discovering Structure in Data Abstract. This paper describes a formal representation of the discovery process that e ciently integrates of any number of data analysis strategies, regardless of their similarities and di erences. We have implemented a system based on this formalization, called the Scientist's Empirical Assistant (SEA). SEA employs several analysis strategies from the discovery literature, including techniques for function nding, causal modeling, and Bayesian conditioning. It uses high-level knowledge about the discovery process, the strategies, and the domain of study to coordinate the selection and application of analyses. It relies on the skills and initiatives of an expert user to guide its search for structure. Finally, it designs and runs experiments with a simulator to verify its ndings. SEA is currently capable of performing a full cycle of discovery and analysis, from hypothesis formulation to experiment design, data collection, analysis, and the generation of explanatory hypotheses. In addition, the formalization on which it is based provides an environment for studying the how, what, why, and when of intelligent data analysis. This paper describes a formal representation of the discovery process that e ciently integrates of any number of data analysis strategies, regardless of their similarities and di erences. We have implemented a system based on this formalization, called the Scientist's Empirical Assistant (SEA). SEA employs several analysis strategies from the discovery literature, including techniques for function nding, causal modeling, and Bayesian conditioning. It uses high-level knowledge about the discovery process, the strategies, and the domain of study to coordinate the selection and application of analyses. It relies on the skills and initiatives of an expert user to guide its search for structure. Finally, it designs and runs experiments with a simulator to verify its ndings. SEA is currently capable of performing a full cycle of discovery and analysis, from hypothesis formulation to experiment design, data collection, analysis, and the generation of explanatory hypotheses. In addition, the formalization on which it is based provides an environment for studying the how, what, why, and when of intelligent data analysis.